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FOREWORD 


This  research  and  development  was  guided  by  Navy  Decision  Coordinating  Paper 
(NDCP)  Z0828-PN  (Enlisted  Personnel  Individualized  Career  System  (EPICS),  formerly 
entitled,  Performance  Aids  Test  and  Evaluation)  under  the  sponsorship  of  the  Deputy 
Chief  of  Naval  Operations  for  Manpower,  Personnel,  and  Training  (OP-01).  The  objectives 
of  the  NDCP  are  to  def:  ie  the  state  of  the  a't  in  job  performance  aid  (JPA)  technology, 
develop  a  conceptual  model  for  an  integrated  JPA-based  personnel  system  including  cost 
benefits  and  tradeoff  analysis,  test  the  JPA  concept,  and  quantify  performance  incre¬ 
ments  and  costs  benefits  obtainable  for  various  applications. 

This  report  is  the  tenth  in  a  series  of  Navy  Personnel  Research  and  Development 
Center  reports  dealing  with  JPA  technology  development:  (1)  NPRDC  TR  77-33  includes 
seven  papers  assessing  the  state  of  the  art  in  3 PA  technology,  ,(2)  NPRDC  TN  78-6 
describes  a  preliminary  enlisted  personnel  system  concept  with  major  emphasis  on  the  use 
of  JPAs,  (3)  NPRDC  TR  78-26  is  a  systematic  review  and  organization  of  existing  JPA 
techniques,  related  research  data,  and  various  applicable  principles  and  concepts,  (4) 
NPRDC,  TN  79-1  defines  a  JPA  selection  algorithm  for  an  integrated  personnel  system,  (5) 
NPRDC  TR .  79-23  discusses  development  of  hybrid  and  enriched  hybrid  troubleshooting 
JPAs,  (6)  NPRDC  TR  82-7  describes  the  development  and  test  of  a  troubleshooting  aid  for 
digital  systems,  (7)  NPRDC  SR  83-32  describes  a  field  evaluation  of  enriched  hybrid 
troubleshooting  JPAs,  (8)  NPRDC  SR  83-39  presents  12  papers  on  factors  influencing  JPA 
cost  and  methods  for  measuring  and  predicting  these  cost  factors,  and  (9)  NPRDC  TR  85- 
24  details  enriched  hybrid  JPA  development.  The  purpose  of  the  present  effort  was  to 
describe  how  fully  and  partially  procedural  zed  JPAs  developed  for  the  EPICS  test  and 
evaluation  were  accepted  and  used  by  EPICS  participants.  Results  are  intended  for  the 
JPA  and  technical  documentation  community. 

Appreciation  is  expressed  to  Mr.  Robert  Bowles,  Naval  Ship  Weapon  Systems 
Engineering  Station,  Port  Hueneme,  California,  who  was  instrumental  in  providing 
technical  data  for  JPA  development  and  who  assisted  directly  in  JPA  validation. 


B.  E.  BACON 
Captain,  U.S.  Navy 
Commanding  Officer 


J.  W.  TWEEDDALE 
Technical  Director 


Avnll.ibUity  Cr  i.'s 


v 


„  e  A.  w  o 

C  °  c 

SUMMARY 

Problem  and  Background 

In  the  maintenance  environment,  technicians  perform  specific  maintenance  tasks, 
which  are  mostly  procedural  and  usually  supported  with  some  kind  of  technical  documen¬ 
tation.  One  form  of  technical  documentation  that  has  been  shown  to  be  effective  is  the 
job  performance  aid  (JPA).  JPAs  are  proceduralized  instructions  that  have  been  used  to 
reduce  training  time,  facilitate  successful  task  performance,  and  increase  utilization  of. 
inexperienced  personnel.  Although  data  on  JPA  use  have  been  positive,  JPAs  must  be 
fully  integrated  into  a  personnel  system  to  achieve  maximum  effectiveness. 

An  integrated  personnel  systems  approach  can  be  used  to  design  a  career  path  in 
which  JPAs  enable  individuals  with  no  formal  technical  training  to  be  effective  members 
of  a  shipboard  work  center.  Individuals  gain  work  experience  by  using  JPAs  to  support 
task  performance;  thus,  the  training  can  be  designed  to  build  upon  that  experience,  and 
training  time  can  be  reduced.  In  addition,  tasks  can  be  grouped,  and  JPAs  can  be  designed 
to  build  upon  the  training  episodes  and  provide  additional  on-the-job  training  (OJT) 
experience  to  prepare  the  individual  for  the  next  training  episode. 

Enlisted  Personnel  Individualized  Career  System  (EPICS)  is  a  systems  approach  that 
jointly  considers  job  design,  training,  JPAs,  career  advancement,  technical  data,  personnel 
organizational  management,  and  personnel  resources.  Within  EPICS,  duty  at  sea  precedes 
formal  technical  training,  which  is  distributed  over  an  individual's  career.  Job  design  is 
used  to  determine  how  tasks  within  a  job  specialty  can  be  arrayed  over  a  career  such  that 
initial  job  requirements  can  be  met  by  using  JPAs. 

Although  a  comprehensive  test  and  evaluation  is  underway  for  EPICS  as  a  system,  an 
in-depth  evaluation  on  the  effectiveness  of  JPAs  within  a  systems  context  was  not 
included. 

Objective 

The  primary  objective  of  this  effort  was  to  determine  if  JPAs  developed  within  an 
integrated  personnel  system  framework  and  in  accordance  with  the  principles  of  job 
design  (1)  would  be  accepted  as  a  valid  form  of  maintenance  documentation  and  (2)  used 
as  a  source  of  maintenance  information  during  task  performance  by  technicians.  A 
secondary  objective  was  to  compare  the  acceptance  and  use  of  JPAs  by  EPICS  partici¬ 
pants  who  (1)  are  eligible  and  ineligible  for  the  fire  control  technician  (FT)  rating 
(redesignated  FC  as  of  July  1985),  (2)  have  high  and  low  aptitude  scores,  (3)  have  high  and 
low  reading  ability,  and  (4)  are  assigned  to  the  Atlantic  and  Pacific  fleets. 

Approach 

Evaluating  the  effectiveness  of  JPAs  involved  collecting  attitudinal  data,  in  the  form 
of  three  questionnaires  and  a  structured  interview,  from:  the  EPICS  participants,  the 
shipboard  administrators,  and  the  work  center  supervisors.  Data  analysis  varied  according 
to  the  instrument  used. 

Results 

After  EPICS  personnel  had  been  onboard  ship  approximately  15  months,  data  were 
collected  frorr^34  work  center  supervisors,  34  shipboard  administrators,  and  112  EPICS 
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participants.  The  participants  were  grouped  on  the  basis  of  their  FT  eligibility,  Armed 
Forces  Qualification  Test  (AFQT)  scores,  and  reading  level.  In  addition,  they  were 
contrasted  according  to  fleet  assignment.  Findings  are  summarized  according  to 
acceptance,  use,  and  implementation. 

1.  Acceptance.  Shipboard  administrators  and  EPICS  personnel  assessed  the  3PAs  as 
well-designed,  complete  sources  of  technical  information,  but  shipboard  administrators 
preferred  maintenance  requirements  cards  (MRCs)  to  3PAs.  EPICS  personnel  preferred 
using  3PAs  for  first  time  and  infrequently  occurring  tasks,  but  preferred  MRCs  for 
frequently  occurring  tasks. 

2.  Use.  3PAs  were  used  for  all  task  types,  but  more  for  unscheduled  than  scheduled 
tasks.  The  FT-ineligible  group  used  3PAs  more  often  than  did  the  FT-eligible  group  for 
scheduled  tasks.  The  lower  AFQT  score  and  lower  reading-grade-level  groups  used  3PAs 
more  often  than  did  the  higher  AFQT  score  and  higher  reading-grade-level  groups  for 
unscheduled  tasks. 

3.  Implementation.  Less  than  UO  percent  of  the  tasks  performed  included  the  use  of 
a  3PA.  Because  of  apparently  better  implementation  in  the  Atlantic  fleet,  the  Atlantic 
fleet  EPICS  personnel  preferred  3PAs  over  MRCs  for  all  task  types  and  used  3PAs  more 
than  did  the  Pacific  fleet  EPICS  personnel. 

Conclusions 

3PAs  were  accepted  and  used  as  a  source  of  technical  information  during  perfor¬ 
mance  of  maintenance  tasks  by  EPICS  personnel  varying  in  FT  eligibility,  general 
aptitude,  and  reading  ability.  The  3PAs  meet  the  information  needs  of  the  lower 
aptitude/poorer  reader  participants. 

Recommendations 

1.  Regardless  of  existing  maintenance  system  guidelines,  task  frequency  in  addition 
to  skill  and  aptitude  level  of  the  3PA  user  should  be  incorporated  as  a  design  criterion  for 
3PA  development. 

2.  The  following  recommendations  for  future  3PA  implementations  within  an 
integrated  personnel  system  context  are  based  on  experience  gained  from  EPICS: 

a.  3PAs  should  not  compete  with  existing  support  documentation  (e.g.,  MRCs) 
even  if  the  target  audience  has  no  preconceptions  about  which  form  of  maintenance 
documentation  they  prefer. 

b.  3PAs  should  be  completely  installed  as  support  documentation  prior  to  the 
arrival  of  the  target  population. 

c.  Work  center  supervisors  should  support  the  3PA  implementation  process. 

d.  Aptitude,  skill  level,  experience,  task  frequency,  and  task  difficulty  should 
be  integrated  into  3PAs  of  differing  levels  of  detail. 
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INTRODUCTION 


Problem 

In  the  maintenance  environment,  technicians  perform  specific  maintenance  tasks, 
which  are  mcv.ly  procedural  and  usually  supported  with  some  type  of  technical. documen¬ 
tation.  One  iorm  of  technical  documentation  that  has  been  shown  to  be  effective  is  the 
job  performance  aid  (JPA).  JPAs  are  proceduralized  instructions  (Figure  1)  that  have 
been  used  to  (1)  reduce  training  time,  (2)  facilitate  successful  task  performance,  and  (3) 
increased  utilization  of  inexperienced  personnel  (Booher,  1977}  Rowan,  1973;  Shriver  & 
Hart,  1975).  However,  when  3PAs  have  been  added  to  an  existing  personnel  system 
withbut  integrating  all  components  of  that  system,  there  have  been  problems  in  both 
acceptance  and  use  of  JPAs  (Johnson,  Thomas,  &  Martin,  1977). 


Objective 


The  primary  objective  of  this  effort  was  to  determine  if  JPAs  developed  within  an 
integrated  personnel  system  framework  and  in  accordance  with  the  principles  of  job 
design  (1)  would  be  accepted  as  a  valid  form  of  maintenance  documentation  and  (2)  used 
source  of  maintenance  information  during  task  performance  by  technicians.  A 
secondary  objective  was  to  compare  the  acceptance  and  use  of  JPAs  by  Enlisted  Personnel 
Individualized  Career  System  (EPICS)  participants  who  (1)  are  eligible  and  ineligible  for 
the  fire  control  technician  (FT)  rating,  (2)  have  high  and  low  aptitude  scores,  (3)  have  high 
and  low  reading  ability,  and  (4)  are  assigned  to  the  Atlantic  and  Pacific  fleets. 


Background 


Although  initially  a  JPA  was  any  device  that  facilitates  the  performance  of  a  given 
task,  the  more  restrictive  definition  now  used  is  a  set  of  step-by-step  instructions 
supported  by  illustrations  providing  complete  start-to-finish  information  for  performing  a 
given  task.  The  need  for  JPAs  became  apparent  when  effective  maintenance  performance 
required  too  much  time  and  too  many  resources.  Foliey  (1961)  noted  that  individuals  with 
an  average  to  high  general  aptitude  had  to  go  through  extensive  training  prior  to  becoming 
successful  maintenance  technicians  and  that  technicians  had  to  retain  an.  extensive 
amount  of  information  in  order  to  perform  various  tasks.  With  JPAs,  technical 
information  could  be  presented  in  a  format  that  (1)  reduces  training  time  (Elliott  <5c  Joyce, 
1971),  (2)  facilitates  performance  by  decreasing  performance  time  and  errors  (Elliott  & 
Joyce,  1971;.  Gebhard,  1970),  and  (3)  allows  lower  aptitude  individuals  to  complete  tasks 
successfully  (Elliott,  1967). 


The  Air  Force  conducted  one  of  the  most  extensive  research  and  development  efforts 
to  demonstrate  the  benefits  of  JPAs.  This  study,  the  Presentation  of  Information  for 
Maintenance  and  Operation  (PIMO)  project  (Serendipity,  1969),  demonstrated  that  inex¬ 
perienced  technicians  with  minimal  technical  training  and  experienced  specialists 
performed  similar  maintenance  tasks  in  about  the  same  time.  Although  project  PIMO  and 
the  design  specifications  that  grew  out  of  it  were  milestones  in  JPA  technology,  other 
elements  in  the  personnel  system  (e.g.,  job  design,  type  of  training,  etc.)  were  not 
considered. 


By  1977,  the  Office  of  the  Secretary  of  Defense,  through  formal  management 
meetings,  determined  that  there  was  a  need  to  (I)  integrate  existing  JPA  technologies 
with  technical  data,  training,  and  personnel  support  systems  and  (2)  demonstrate  the 
feasibility  of  an  integrated  personnel  systems  approach.  As  a  result,  Foley  (1978) 
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13.  Position  new  power  supply  (15,  16,  18)  onto  mounting  plate 
(23).  Install  four  screws,  lockwashers,  and  washers  (25). 

14.  Position  new  power  supply  (17)  onto  mounting  plate  (23). 
Install  six  Phillips  screws,  lockwashers,  and  washers  (26). 


0 


Figure  1.  An  example  from  a  job  performance  aid  illustrating  the 
typical  format. 
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emphasized  JPA  technology  in  developing  a  model  to  provide  productive  personnel  without 
large  front-end  training  costs.  The  model,  however,  did  not  adequately  address  job  design 
or  its  use  to  structure  an  individualized  career  path  based  upon  specific  system 
requirements.  One  effort  that  integrated  JPA  technology  with  all  other  elements  of  a 
personnel  system  was  EPICS. 

EPICS  emphasizes  the  use  of  job  design  in  the  design  and  development  of  JPAs 
(Blanchard  &  Smillie,  1980).  EPICS  is  the  product  of  an  integrated  personnel  systems 
approach  that  considers  job  design,  training,  OP  As,  career  advancement,  technical  data, 
personnel  organizational  management,  and  personnel  resources.  Tradeoffs  among  these 
seven  elements  resulted  in  the  development  of  a  six-year  model  comprising  three 
successive  skill  levels.  The  NATO  Seasparrow  Surface  Missile  System  (NSSMS),  which  is 
operated  and  maintained  by  FTs,  was  selected  as  the  test  system.  Blanchard,  Smillie,  and 
Conner  (1984)  describe  the  EPICS  design,  development,  and  implementation  in  depth. 

EPICS  defers  formal  technical  training  and  distributes  it  over  an  individual's  career. 
In  addition,  job  design  is  used  to  determine  how  tasks  can  be  arrayed  over  the  career  so 
that  initial  job  requirements  can  be,  met  using  JPAs. 

A  task  and  behavioral  analysis  determined  that  two  types  of  JPAs,  fully  and  partially 
proceduralized,  were  needed  to  support  maintenance  performance.  Fully  proceduralized 
JPAs  (FPJPAs)  were  designed  for  EPICS  personnel  who  were  in  the  first  year  of  their 
career  and  had  no  training  or  experience  in  performing  the  various  maintenance  tasks. 
Based  on  previous  research  (e.g.,  project  PIMO)  FPJPAs  were  designed  as  complete,  very 
detailed  sets  of  maintenance  procedures  that  focus  on  illustrations  for  the  task  steps  and 
use  brief  narrative  statements  .only  to  support  the  illustration  (Figure  2).  Partially 
proceduralized  JPAs  (PPJPAs)  were  designed  for  EPICS  personnel  who  had  approximately 
one  year  of  shipboard  technical  experience  and  some  basic  electronics  training.  Thus, 
PPJPAs  relied  more  heavily  on  narrative  descriptions  of  task  steps.  In  contrast  to  the 
FPJPAs,  illustrations  were  used  only  for  maintenance  tasks  areas  that  personnel  had  not 
performed  many  times  during  their  first  year  in  the  Navy  (Figure  3). 

The  comprehensive  test  and  evaluation  designed  for  EPICS  as  a  system  (Blanchard, 
Clelland,  <5c  Megrditchian,  1984)  did  not  include  an  in-depth  evaluation  of  JPA 
effectiveness  within  the  system's  context.  Specifically,  while  JPAs  have  been  shown  to  be 
effective  adjuncts  to  existing  technical  documentation  and  training  for  fielded  systems, 
how  JPAs  will  be  accepted  and  used  when  developed  in  accordance  with  an  integrated 
personnel  system  is  not  known. 


Subjects 


APPROACH 


After  EPICS  personnel  had  been  onboard  ship  approximately  15  months,  data 
describing  various  characteristics  of  JPAs  were  collected  from  EPICS  participants, 
shipboard  administrators,  and  work  center  supervisors.  The  112  EPICS  participants  were 
aboard  34  ships,  had  recently  completed  the  first  skill  level  in  the  EPICS  career  path 
(apprentice  technician  duty),  had  JPAs  available  as  guidance  for  scheduled  and  unsched¬ 
uled  maintenance  tasks,  and  had  completed  the  shipboard  training  phase  prior  to  attending 
the  first  shore-based  school.  Of  these  112,  27  were  interviewed,  110  responded  to  the 
feedback  survey,  70  completed  the  JPA  usage  survey,  and  45  were  rated  by  their  work 
center  supervisor.  Most  of  the  EPICS  personnel  had  been  in  the  Navy  approximately  16 
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Figure  2.  An  extract  from  an  EPICS  fully  proceduralized  3PA  (FP3PA). 
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Figure  3.  An  extract  from  an  EPICS  partially  proceduralized  3PA  (PP3PA). 


months  and  ranged  in  pay  grade  from  E-2  to  E-4.  Table  1  provides  demographic 
characteristics  of  each  group  of  respondents. 


Table  1 

Demographic  Characteristics  for  EPICS  Respondent  Groups 


Demographics 

EPICS 

Ships 

Fleet8 

FT  h 
Eligibility0 

AFQTG 

Score 

Reading 

Grade 

Level 

Education 

Levei 

(Years) 

Group 

(N=34) 

(%) 

A 

(96) 

P 

(%) 

E 

(%) 

I 

(96) 

X 

SD 

X 

SD 

X 

SD 

Totald 

(N=>112) 

100 

51 

49 

52 

48 

66 

17.3 

11 

1.8 

12 

.8 

Interviews 

(N=27) 

38 

58 

42 

«« 

•  _ 

.  . 

•  • 

«•»  m 

•  • 

-- 

Feedback 

Survey 

(N=U0) 

100 

50 

50 

52 

48 

66 

17.2 

11 

1.9 

12 

.8 

JPA  Usage 
Survey 
(N=70) 

79 

70 

30 

43 

57 

64 

17.8 

11 

1.8 

12 

.6 

Supervisor 

Confidence 

Ratings 

(N=45) 

68 

71 

29 

56 

44 

63 

18.2 

11 

1.7 

12 

.7 

aA  =  Atlantic  fleet;  P  =  Pacfic  fleet. 

=  eligible  for  fire  control  technician  rating;  I  =  ineligible  for  FT  rating. 

CAFQT  =  Armed  Forces  Qualification  Test;  X=  mean. 

dTotal  number  of  EPICS  personnel  completing  apprentice  technician  duty  (have  received 
no  formal  shore-based  training). 


Of  the  34  shipboard  administrators,  10  were  interviewed  and  17  responded  to  the 
feedback  survey.  To  preserve  anonymity,  few  personal  demographics  were  obtained  from 
the  shipboard  administrators.  Most  had  been  in  the  Navy  4  to  5  years  and  were  generally 
E-5s,  although  a  few  were  more  senior. 

The  34  work  center  supervisors  were  requested  to  rate  their  confidence  in  EPICS 
personnel;  29  completed  rating  forms  for  45  EPICS  personnel.  Demographics  on  the 
supervisors  were  similar  to  those  of  the  shipboard  administrators. 


Instruments 


Four  different  instruments  were  developed  to  obtain  data:  feedback  surveys,  JPA 
usage  {survey,  supervisor  confidence  ratings,  and  structured  interviews.  These  instruments, 
provided  information  on  JPA  acceptance  and  use.  Table  2  provides  the  sample  size  and 
response  rate  for  each  measurement  instrument. 

Table  2 

Variable/Measure  Relationships  and  Measure  Response  Rates 


Variables 

Measure 

Sample  Size 

Response  Rate3  (%) 

Acceptance 

EPICS  personnel  feedback 
survey 

110  . 

98 

Shipboard  administrator 
feedback  survey 

17 

50 

JPA  usage  survey 

70b 

62 

EPICS  personnel  interviews 

27 

24 

Shipboard  administrator 
interviews 

10 

29 

Use 

JPA  usage  survey 

70b 

62 

Supervisor  confidence  ratings 

45C 

40 

^PICS  personnel  return  rates  are  based  on  total  N  =  112;  shipboard  administrator 
response  rates  on  total  N  s  34. 

bOnly  EPICS  personnel  completed  the  JPA  usage  survey, 
c 

This  sample  size  is  based  on  the  number  of  EPICS  participants  for  whom  both  supervisor 
rating  data  and  JPA  usage  data  were  available. 


1.  Feedback  surveys.  As  part  of  the  overall  evaluation  of  EPICS,  a  series  of 
feedback  surveys  was  developed  to  assess  the  attitudes  of  EPICS  personnel  and  shipboard 
administrators.  Only  responses  related  to  JPA  items  are  reported  here.  (Blanchard, 
Clelland,  4  Megrditchian,  1984),  summarize  the  data  on  other  EPICS  variables.)  The  JPA 
questions  on  the  feedback  survey  addressed  JPA  acceptance:  Shipboard  administrators 
were  given  9  questions  and  EPICS  sailors  were  give*  16  questions.  Each  question  started 
with  "To  what  extent"  and  had  a  3-point  Likert  response  scale.  The  scale  ranged  from  1 
(very  little  extent)  to  3  (very  great  extent). 

2.  JPA  usage  survey.  The  JPA  usage  survey  was  administered  to  EPICS  personnel 
when  they  arrived  at  their  first  school,  equipment  technician  training  (ETT).  The  survey 
consisted  of  six  questions  on  JPA  accep lance  and  listed  the  198  tasks  for  vhich  JPAs  had 
been  developed.  Since  both  JPAs  and  maintenance  requirement  cards  (MRCs)  were 
available  for  these  tasks,  the  questions  asked  which  type  of  maintenance  documentation 
the  EPICS  personnel  prefer.  (An  MRC  is  an  abbreviated  checklist  with  no  illustrations.) 
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For  the  task  listing,  respondents  were  instructed  to  estimate  the  number  of  times 
they  had  performed  each  task  and  the  number  of  times  they  had  used  a  JPA  to  perform 
each  task.  Although  the  usage  survey  provided  some  information  on  JPA  acceptance,  its 
primary  use  was  to  obtain  information  on  the  frequency  of  JPA  use.  Table  1  shows  70 
percent  of  those  responding  to  the  JPA  usage  survey  were  assigned  to  the  Atlantic  fleet, 
approximately  20  percent  more  than  in  the  initial  EPICS'  cohort  distribution  (Blanchard, 
Smillie,  &  Conner,  1984,  Table  3,  p.  10). 

3.  Supervisor  confidence  ratings.  Supervisor  confidence  ratings  rated  the  degree  of 
confidence  a  supervisor  had  in  the  ability  of  EPICS  personnel  to  perform  maintenance 
tasks.  Supervisor  confidence  was  assumed  to  be  related  to  JPA  use.  The  following 
technical  areas  that  are  relevant  to  JPA  usage  were  rated:  (a)  clean,  inspect,  and 
lubricate  tasks,  (b)  remove  and  replace  tasks,  (c)  system  tests,  (d)  adjustment  of  system 
components,  (e)  ordnance  handling,  (f)  fault  isolation,  and  (g)  operator  tasks.  EPICS 
shipboard  administrators  were  instructed  to  rate  their  confidence  in  each  EPICS  partici¬ 
pant  in  performing  each  task  according  to  the  degree  of  assistance  required.  The  scale 
used  ranged  from  1  (not  at  all  confident;  always  required  assistance)  to  5  (extremely 
confident;  never  required  assistance)  with  0  (not  observed, ,  so  cannot  assess)  and  9 
(participant  has  not  performed)  also  possible.  The  data  were  collected  when  the  EPICS 
personnel  completed  apprentice  technician  duty.  As  with  the  JPA  usage  survey,  71 
percent  of  those  responding  were  assigned  to  the  Atlantic  fleet  (see  Table  1). 

4.  Structured  interviews.  Structured  interviews  of  both  EPICS  personnel  and 
shipboard  administrators  covered  a  variety  of  topics  including  preference  for  JPAs  or 
MRCs  as  the  technical  documentation  to  support  shipboard  maintenance.  Only  the  JPA 
and  MRC  related  responses  are  reported  here. 

Analysis 

Data  were  collected  only  for  the  initial  EPICS  skill  level,  where  FPJPAs  are  supposed 
to  be  used,  and  not  for  the  advanced  skill  levels,  where  PPJPAs  are  supposed  to  be  used. 
Consequently,  the  data  are  only  representative  of  FPJPAs. 

descriptive  statistics  were  computed  for  both  shipboard  administrators  and  EPICS 
personnel  on  the  feedback  survey  items  related  to  acceptance.  Response  percentages 
were  computed  for  each  point  of  the  5-point  survey  scale.  Data  from  the  EPICS  personnel 
were  grouped  according  to  the  following  four  variables  and  t-tests  were  computed  to  test 
for  differences  between  groups:  (1)  FT  eligibility,  (2)  AFQT  score,  (3)  reading  grade  level, 
and  (4)  fleet  assignment. 

Eligibility  for  the  FT  rating  is  based  on  a  composite  score  of  specific  subtests  on  the 
Armed  Forces  Vocational  Battery  (ASVAB).  Individuals  with  a  composite  score  of  218  or 
greater  are  eligible  to  become  FTs;  those  with  less  than  218  are  ineligible. 

AFQT  is  a  general  aptitude  screening  criterion  that  is  based  on  a  different  composite 
score  of  subtests  on  the  ASVAB.  Data  were  collected  from  40  EPICS  personnel  with  a  low 
AFQT  score  (less  than  or  equal  to  60)  and  30  with  a  high  AFQT  score  (more  than  60). 

Reading  grade  level  is  defined  as  a  measure  of  reading  ability  as  determined  by  the 
Gates-McGinitie  Reading  Test.  Data  were  collected  from  25  participants  with  a  low 
reading  grade  level  (below  grade  11)  and  from  44  with  a  high  reading  grade  level  (grade  11 
or  above). 
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Fleet  assignment  is  the  fleet  command  of  ships  to  which  the  EPICS  personnel  were 
assigned.  Data  were  collected  from  49  personnel  assigned  to  the  Atlantic  fleet  and  21 
assigned  to  the  Pacific  fleet. 

FT  eligibility  was  used  to  divide  EPICS  personnel  to  determine  any  differences  in 
acceptance  and  use  of  3PAs  between  the  FT-eligible  and  FT-ineligible  groups.  AFQT 
scores  were  used  to  determine  if  the  results  categorized  according  to  the  electronics 
aptitude  measure  were  similar  to  this  more  general  aptitude  measure*  Since  the  users  had 
to  "read"  material  in  the  3PA,  reading  grade  level  measures  were  used  to  determine  if 
acceptance  and  use  of  3PAs  by  good  and  poor  readers  differed.  EPICS  personnel  were 
assigned  to  the  Pacific  fleet  approximately  6  months  prior  to  those  assigned  to  the 
Atlantic  fleet.  Comparisons  between  the  two  fleets  were  made  to  determine  if 
differential  3PA  acceptance  or  use  would  result  from  implementation  variations. 

Response  percentages  were  computed  for  the  EPICS  personnel  on  the  six  usage  survey 
questions  related  to  3PA  acceptance.  Chi  square  values  were  also  computed  to  contrast 
groups  on  each  of  the  four  variables. 

The  usage  survey  was  designed  to  provide  performance  information  on  the  198  tasks 
for  which  3PAs  had  been  developed.  Tasks  were  grouped  into  two  categories,  scheduled 
and  unscheduled.  Scheduled  tasks  are  preventive  maintenance  tasks  (cleaning,  adjusting, 
testing)  that  are  performed  periodically  (e.g.,  weekly,  monthly).  Unscheduled  tasks,  on 
the  other  hand,  are  corrective  maintenance  tasks  (remove,  replace)  that  are  performed 
aperiodically  (i.e.,  when  required).  From  a  total  of  198  possible  tasks,  42  were  never 
performed.  Table  3  shows  the  number  of  tasks  in  each  category. 

Table  3 

Number  of  Tasks  by  Performance  Category 


Task  Category 

Performed 

Never  Performed 

Possible 

Scheduled 

46 

0 

46 

Unscheduled 

110 

42 

152 

Total 

156 

42 

198 

Although  technical  documentation,  such  as  JPAs,  is  available,  it  is  not  always  used 
when  a  maintenance  task  is  performed.  For  example,  technicians  who  have  performed  a 
particular  task  a  number  of  times  in  the  past  may  be  confident  that  they  can  presently 
perform  the  task  without  referring  to  the  documentation.  Since  MRCs  were  also 
available,  technicians  may  never  use  a  JPA  for  a  particular  task.  Therefore,  a  usage  ratio 
(3PA  frequency/task  frequency)  was  computed  for  each  task  an  individual  had  performed. 
Usage  ratios  were  then  summed  across  tasks  within  each  of  the  task  categories.  Then,  an 
average  usage  ratio  was  computed  for  each  task  category.  For  example,  a  value  of  0.5  or 
50  percent  indicates  that  EPICS  personnel  used  a  3PA  one  out  of  every  two  times  they 
performed  a  particular  task. 


In  order  to  account  lor  the  tasks  for  which  JPAs  were  never  used,  two  usage  ratios 
were  computed. 

.1.  Implementation  (IMP)  usage  ratios  were  computed  lor  all  tasks  that  EPICS 
personnel  reported  they  had  performed  at  least  once.  For  example,  if  an  individual 
performed  one  scheduled  task  5  times  and  another  scheduled  task  20  times,  all  25  times 
were  counted  regardless  of  whether  JPAs  had  been  used.  If  the  individual  did  not  use  a 
JPA  on  the  first  task  and  used  a  JPA  10  times  for  the  second  task,  the  IMP  value  for  those 
two  tasks  would  be  (0  ♦  10)/(5  +  20)  or  40  percent.  High  IMP  values  would  indicate  that 
JPAs  were  perceived  as  a  part  of  the  official  maintenance  documentation  system. 

2.  Specific  JPA  usage  (USE)  ratios  were  computed  only  for  tasks  for  which  JPAs 
had  been  used  at  least  once.  Using  the  same  example  as  for  IMP,  the  USE  ratio  would  be 
(10/20)  or  50  percent.  The  task  performed  five  times  without  using  the  JPA  does  not 
enter  into  the  calculations.  High  USE  ratios  indicate  that  the  personnel  who  used  the 
JPAs  consider  them  to  be  essential  aids  for  performing  maintenance.  To  determine  if 
USE  raios  would  be  influenced  by  FT  eligibility,  AFQT  score,  or  reading  grade  level,  a 
one-way  analysis  of  variance  was  computed  for  each.  USE  ratios  were  selected  as 
indicators  of  JPA  usage  because  the  reasons  that  JPAs  were  never  used  for  given  tasks 
could  not  be  determined.  In  fact,  it  was  quite  likely  that  no  documentation--neither  JPAs 
nor  MRCs~was  used  for  some  tasks. 

.  Pearson  correlations  were  computed  between  supervisor  confidence  ratings  of  the 
EPICS  personnel  and  the  JPA  usage  measure  (USE  ratio)  for  the  following  technical  task 
areas:  (1)  clean,  inspect,  and  lubricate  tasks,  (2)  remove  and  replace  tasks,  (3)  system 
tests,  and  (4)  adjustment  of  system  components.  Data  were  gathered  from  34  work  center 
supervisors. 

Interview  data  were  content  analyzed  for  all  information  referring  to  some  aspects  of 
JPAs.  These  comments  were  then  categorized  according  to  common  themes. 


RESULTS 


Acceptance 

Although  the  feedback  survey  had  a  5-point  scale  to  allow  respondents  a  range  for 
responses,  the  twQ  upper  and  lower  response  categories  were  collapsed  to  facilitate  survey 
data  interpretation.  Tables  4  and  5  provide  the  response  percentages  of  the  EPICS 
shipboard  administrators  and  the  EPICS  personnel  respectively  fof  each  response  category 
as  well  as  the  means  and  standard  deviations  (SD)  of  the  5-point  scale.  The  t-tests  that 
were  run  of  the  means  for  the  various  EPICS  personnel  groupings  (Table  5)  revealed 
significant  differences  between  personnel: 

1.  With  low  and  high  AFQT  scores  for  Question  16,  t  (17)  =  -2.3,  p  <  .05. 

2.  With  lo  v  and  high  reading  grade  levels  for  Question  9,  t  (S9)  =  -2.6,  p  <  .05. 

3.  With  low  and  high  reading  grade  levels  for  Question  13,  t  (17)  =  3.2,  p  <  .01. 

4.  Assigned  to  the  Atlantic  and  Pacific  fleets  for  Question  1,  t  (108)  =  -2.7,  p  <  .01. 
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5.  Assigned  to  the  Atlantic  and  Pacific  fleets  for  Question  7,  t  (17)  =  2.2,  p  <  .05. 

(.  Assigned  to  the  Atlantic  and  Pacific  fleets  for  Question  10,  t  (108)  =  >2.1, 


Table  4 

EPICS  Shipboard  Administrator  Responses  on  JPA  Design 


Resoonse  Category  (%) 

Question  (N=17) 

Very  Little/ 
Little 

Some 

Great/ 

Very  Great 

Mean  SD 

(5-point  scale) 

To  what  extent . . . 

.  . . . 

. .  . 

_ _ _ _  . 

1.  do  you  have  to  assist 
EPICS  sailors  when 
they  use  JPAs? 

70 

18 

12 

2.1 

1.0 

2.  do  EPICS  sailors 
find  JPAs  easy  to 
follow? 

12 

88 

4.4 

.7 

3.  are  there  errors  in 

JPAs? 

59 

35 

6 

2.1 

1.0 

4.  do  JPAs  contain  all 
the  information  for 
the  job? 

6 

18 

76 

3.9 

.8 

3.  is  it  necessary  to 
have  training  in  the 
use  of  JPAs? 

64 

29 

7 

2.2 

1.3 

6.  do  EPICS  sailors  use 
MRCs  when  JPAs  are 
available? 

12 

18 

70 

3.8 

1.0 

7.  does  having  JPAs  make 
you  more  confident  in 
assigning  EPICS  sailors? 

58 

24 

18 

2.4 

1.1 

8.  are  the  JPAs  overly 
simplified? 

6 

29 

65 

4.1 

1.0 

9.  would  you  be  satisfied 
with  JPAs  for  all 
maintenance? 

94 

6 

1.4 

1.0 

11 


Table  5 


EPICS  Personnel  Responses  on  3PA  Design 


Response  Category  (%) 

Question 

N 

Very  Little/ 
Little 

Some 

Great/ 
Very  Great 

Mean  SD 
(5-point  scale) 

To  what  extent . . . 

1.  do  you  need  help  when 
using  3PAs? 

110 

87 

9 

4  . 

1.6 

.8 

2.  do  3PAs  contain  all 
the  information  for 
a  job? 

110 

4 

26 

70 

3.8 

.9 

3.  are  you  asked  to  show 
EPICS  people  how  to 
use  3PAs? 

19 

67 

24 

9 

1.9 

1.0 

4.  are  3PAs  easily  lost? 

19 

100 

-- 

-- 

1.5 

.5 

5.  do  you  have  difficulty 
getting  3PAs? 

19 

68 

23 

9 

2.1 

1.2 

6.  do  3PA  pages  get  lost, 
torn,  or  dirty? 

19 

96 

4 

•  • 

1.4 

.6 

7.  are  3PAs  easy  to  handle 
while  working? 

19 

15 

45 

40 

3.2 

1.0 

8.  is  it  necessary  to  have 
training  in  the  use  of 
3PAs? 

110 

84 

10 

6 

1.7 

.9 

9.  are  you  asked  to  show 
others  how  to  use  3PAs? 

91 

74 

15 

11 

1.9 

1.1 

10.  are  the  3PAs  hard  to 
understand? 

110 

92 

6 

2 

1.5 

.7 

11.  are  the  3PAs  correct? 

no 

10 

38 

52 

3.4 

.9 

12.  are  the  3PA  pictures 
hard  to  follow? 

no 

86 

11 

3 

1.6 

.8 

13.  are  the  3PAs  overly 
simplified? 

19 

36 

40 

24 

2.8 

1.0 

14.  would  you  be  satisfied 
using  3PAs  for  all 
maintenance? 

110 

37 

40 

23 

2.7 

1.2 

15.  would  you  want  to  have 
3PAs  if  you  changed 
jobs? 

110 

26 

48 

26 

3.0 

1.0 

16.  do  3PAs  help  you  be¬ 
come  a  useful  member 
of  your  work  center? 

19 

28 

20 

52 

3.2 

1.2 

12 

-  -v  \  rw  \ 

\r*  ,..iv  v 

>  -w  u>i  _  «•  .  ^ 

v  •  v  •»  -um  v  m  * 

»  V  *  «  ■  v. 

Table  6  gives  the  responses  of  EPICS  personnel  to  the  first  four  questions  of  the  JPA 
usage  survey.  These  questions  are  related  to  JPA  preference  (i.e.,  how  well  the  EPICS 
participants  liked  the  JPAs).  Only  one  significant  statistical  difference  was  found 
between  EPICS  personnel  assigned  to  the  Atlantic  and  Pacific  fleets  for  infrequently 
occurring  tasks  (Question  4),  chi  square  3  7.2,  p  <  .05). 

Table  6 

EPICS  Personnel  Responses  on  JPA  Preference 


ResDonse  (%)' 

1 

Question 

N 

Yes  for 
Most  Jobs 

Yes  for 
Some  Jobs 

No,  I  prefer 
to. use  an  MRC 

1. 

Do  you  like  to  use  a  JPA  the 
first  time  you  do  a  scheduled 
maintenance  task? 

69 

33 

44 

22 

2. 

Do  you  like  to  use  a  JPA  the 
first  time  you  do  an  unscheduled 
maintenance  task? 

69 

47 

34 

16 

3. 

Do  you  like  to  use  JPAs  for 
frequently  occurring  maintenance 
tasks? 

69 

6 

23 

70 

4. 

Do  you  like  to  use  JPAs  for 
infrequently  occurring  main- 
tenance  tasks? 

69 

46 

34 

17 

al  to  3  percent  did  not  respond. 


Specific  comments  related  to  JPA  acceptance  from  the  open-ended  questions  of  the 
interview  and  JPA  usage  survey  were  grouped  according  to  whether  they  were  stated  in 
terms  of  advantages  or  disadvantages  of  JPAs.  Table  7  provides  representative  samples 
of  EPICS  shipboard  administrator  comments;  Table  8  representative  samples  from  the 
EPICS  personnel.  Table  9  summarizes  the  responses  from  the  two  open-ended  questions 
of  the  JPA  usage  survey. 

Use 


Results  from  the  JPA  usage  survey  are  grouped  according  to  whether  or  not  the  tasks 
were  scheduled.  Forty-six  scheduled  tasks  were  performed  13,510  times  with  JPAs  used 
*,086  times.  Similarly,  110  unscheduled  tasks  were  performed  3,042  times  with  only  88 
performed  using  JPAs— for  a  total  of  721  times. 
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Table  7 


EPICS  Shipboard  Administrator  Interview  Responses 


Advantages  of  JPAs 


1.  Personnel  can  use  3PAs  to  provide  location  information  and  maintenance  procedures 
for  areas  that  are  unfamiliar  to  them.  (N=10) 

2.  Good  for  infrequently  occurring  tasks  or  when  unfamiliar  with  a  particular  task. 
(N=3) 

3.  Excellent  for  location  information.  (N=3) 


Disadvantages  of  JPAs 

1.  Too  simple.  (N=9) 

2.  .  JPAs  arrived  after  the  EPICS  sailors;  used  MRCs  instead.  (N=7) 


Table  8 

EPICS  Personnel  Interview  Responses 


Advantages  of  3PAs 

1.  They're  great  when  you  first  come  aboard.  (N=15) 

2.  Location  illustrations  help  when  not  familiar  with  equipment.  (N=12) 

3.  Could  have  gotten  along  without  them  initially,  but  would  have  required  much  more 
supervision.  (N=ll) 

4.  Use  JPAs  the  first  couple  of  times  to  do  a  task  and  until  I  feel  confident  enough  to 
use  MRC.  (N=3) 


Disadvantages  of  3PAs 


1.  JFAs  did  not  arrive  until  2  to  3  months  after  we  were  using  MRCs.  (N=4) 

2.  Didn't  need  them;  always  had  someone  supervising  us  the  first  couple  of  times.  (N=4) 
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Table  9 


EPICS  Personnel  Responses  to  Open-ended  Questions  of 
JPA  Usage  Survey 


What  are  your  reasons  for  using  a  3 PA  instead  of  an  MRC  for  a  task? 

1.  When  doing  a  task  for  the  first  time  or  unfamiliar  with  the  equipment.  (N=13) 

2.  Do  not  always  understand  what  has  to  be  done  after  reading  the  MRC;  the  JPA  shows 
the  steps  that  have  to  be  done.  (N=10) 

3.  On  difficult  tasks;  the  JPA  is  more  detailed;  less  likely  to  make  any  mistakes.  (N=7) 

4.  JPAs  explain  things  better.  (N=7) 

3.  JPAs  are  used  with  MRCs  for  a  better  understanding  of  the  task.  (N=7) 

6.  JPA  will  make  the  MRC  easier  to  understand  if  it  is  a  difficult  task.  (N=5) 

7.  Sometimes  can't  get  senior  people  to  show  where  something  isJocated;  the  JPAs  do. 
(N=3) 

8.  Don't  know  enough  about  the  task.  (Nr3) 

9.  When  doing  an  infrequent  or  difficult  task,  the  JPAs  leave  little  room  for  error. 
(N=2) 

10.  MRCs  are  hard  to  understand  when  doing  a  task  the  first  time.  (N=l) 

11.  JPAs  keep  the  task  sequence  organized.  (Nsl) 

What  are  your  reasons  for  using  an  MRC  instead  of  a  JPA  for  a  task? 

1.  If  you  are  already  familiar  with  the  procedures  for  a  particular  task,  using  an  MRC  is 
faster.  (N=12) 

2.  MRC  is  quicker,  cuts  down  to  the  basic  steps.  (N=ll) 

3.  For  many  tasks,  an  MRC  has  enough  information  on  it. to  complete  the  task.  (N=7) 

4.  On  less  complicated  tasks,  an  MRC  is  better  because  it  doesn't  give  you  unnecessary 
information.  (N=6) 

3.  Most  times  when  JPAs  are  not  used,  a  petty  officer  was  usually  verbally  instructing 
while  reading  from  the  MRC.  (N=5) 

6.  MRCs  are  easier  to  read  and  understand  than  JPAs.  (N*5) 

7.  Already  using  MRCs  when  JPAs  arrived.  (N=5) 

8.  For  many  tasks,  the  JPAs  are  too  simple.  (N=4) 
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Figure  4  gives  the  IMP  ratios  for  the  scheduled  and  unscheduled  tasks.  The  number  of 
respondents  who  performed  each  task  group  is  also  given.  Figure  5  breaks  down  the  IMP 
values  by  fleet  assignment.  One-way  analyses  of  variance  were  computed.  The  only 
significant  differences  were  between  EPICS  personnel  assigned  to  the: 

1.  Atlantic  and  Pacific  fleets  for  scheduled  tasks  F  (1,  68)  =  7.2,  p  <  .01. 

2.  Atlantic  and  Pacific  fleets  for  unscheduled  tasks  F  (1,  68)  =  5.8,  p  <  .05. 

Figure  6  gives  the  overall  USE  ratios  for  scheduled  and  unscheduled  tasks.  Figures  7 
through  9  break  down  the  USE  by  FT  eligibility,  AFQT  score,  and  reading  grade  level. 
One-way  analyses  of  variance  were  computed.  The  only  significant  differencels  in  USE 
ratios  were  found  between: 

1.  FT -eligible  and  FT-ineligible  personnel  for  scheduled  tasks  F  (I,  59)  =  5.6, 
p  <  .05. 

2.  Personnel  with  low  and  high  AFQT  scores  for  scheduled  tasks  F  (1,  59)  =  8.8, 
p  <  .05. 

3.  Personnel  with  low  and  high  AFQT  scores  for  unscheduled  tasks  F  (1,  42)  =  4.5, 
p  <  .05. 

4.  Personnel  with  low  and  high  reading  grade  levels  for  unscheduled  tasks  F  (1,  42)  = 
2.6,  p  <  .05. 

Table  10  provides  the  correlation  coefficients  between  supervisor  confidence  ratings 
and  JPA  USE  ratios  by  group  and  technical  task  area. 


DISCUSSION 


Although  the  primary  purpose  of  this  study  was  to  assess  how  well  maintenance 
technicians  accept  and  use  JPAs,  information  was  also  obtained. on  how  well  the  JPAs 
were  integrated  into  the  NSSMS  maintenance  documentation.  Thus,  the  discussion  is 
divided  into  three  parts: 


1.  Acceptance. 

2.  Use— the  degree  to  which  JPAs  were  used  for  particular  tasks. 

3.  Implementation— the  degree  to  which  JPAs  were  implemented  as  part  of  the 
Navy  maintenance  system. 

Acceptance 

The  majority  of  EPICS  shipboard  administrators  indicated  that  the  JPAs  were  easy 
for  EPICS  participants  to  follow  (88%),  contained  few  errors  (59%),  contained  all  the 
information  to  do  the  job  (76%),  and  did  not  inquire  much  training  to  be  used  (64%)  and 
that  EPICS  personnel  did  not  require  much  assistance  when  performing  a  task  using  JPAs 
(70%).  On  the  other  hand,  the  majority  of  shipboard  administrators  (70%)  stated  that 
EPICS  personnel  prefer  to  use  the  conventional  documentation— MRCs— rather  than  JPAs 
for  a  given  task  (70%),  they  were  not  more  confident  in  assigning  EPICS  personnel  with 
JPAs  (58%),  JPAs  were  overly  simplified  (65%),  and  they  would  not  be  satisfied  having  to 
use  JPAs  for  all  maintenance  (94%). 
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IMP  RATIO  (%) 


SCHEDULED* 


UNSCHEDULED* 


TASK  CATEGORY 

‘Performed  by  70  EPICS  participant*. 


Figure  4.  Overall  JPA  implementation  (IMP)  ratios  by  task  category. 
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SCHEDULED* 


UNSCHEDULED* 


TASK  CATEGORY 


‘Performed  by  <i9  Atlantic  fleet  and  21  Pacific  fleet  EPICS  participants. 


Figure  5.  IMP  ratios  by  fleet  assignment  and  task  category 
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“Performed  by  61  EPICS  participant!. 
bPerformed  by  »*  EPICS  participant!. 


Figure  6.  USE  ratios  by  task  category. 
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“Performed  by  31  FT-ineligibie  and  26  FT-eligible  EPICS  participants. 
^Performed  by  21  FT -ineligible  and  19  FT-eligible  EPICS  participants. 


Figure  7.  USE  ratios  by  FT  eligibility  and  task  category 
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"Performed  by  23  EPICS  participants  with  low  AFQT  scores  and  34  with  high  AFQT 
scores. 

bPerformed  by  IS  EPICS  participants  with  low  AFQT  scores  and  26  with  high  AFQT 
scores. 


Figure  8.  USE  ratios  by  AFQT  scores  and  task  category. 


TASK  CATEGORY 

"Performed  by  22  EPICS  participants  with  low  reading  grade  levels  and  39  with  high 
reading  grade  levels. 

^Performed  by  IS  EPICS  participants  with  low  reading  grade  levels  and  26  with  high 
reading  grade  levels. 


Figure  9.  USE  ratios  by  reading  grade  level  and  task  category 


Correlation  Coefficients  Between  Supervisor  Confidence 
Ratings  and  JPA  USE  Ratios 


Technical  Task  Area 

Task 

Clean  and 

Remove  and 

System 

Croup 

Inspect 

Replace 

Tests 

Adjustment 

Scheduled 

-.39* 

(N=38) 

-.12 

(N=27) 

.12 

,(N=38) 

-.23 

(N=29) 

Unscheduled 

-.10 

(N=28) 

.24 

(N=22) 

.10 

(N=28) 

.08 

(N=22) 

*p  a  <  .01. 


While  the  shipboard  administrators  were  divided  in  their  overall  acceptance  of  JPAs, 
the  EPICS  personnel's  acceptance  was  generally  positive  or  at  least  ambivalent.  Compar¬ 
ing  the  answers  to  two  questions  that  were  the  same  for  both  the  shipboard  administrators 
and  the  EPICS  personnel,  only  24  percent  of  the  EPICS  personnel  stated  that  the  JPAs 
were  overly  simplified  and  only  37  percent  would  be  dissatisfied  with  using  JPAs  for  all 
maintenance,  while  the  shipboard  administrators  answered  65  and  94  percent  respectively. 
This  difference  in  responses  is  not  surprising  considering  that  the  EPICS  personnel  were 
inexperienced  technicians  and  were  less  likely  to  resist  changing  from  MRCs  to  JPAs. 

In  comparing  the  feedback  survey  responses  of  the  groups  of  EPICS  personnel,  there 
were  some  differences.  The  low  AFQT  score  group  was  more  positive  about  the  JPAs 
helping  them  become  useful  members  of  the  work  center.  The  low  reading  grade  level 
group  (1)  was  asked  least  often  to  show  others  how  to  use  JPAs  and  (2)  was  less  negative 
about  the  JPAs  being  overly  simplified.  Johnson  et  al.  (1977)  also  found  that  JPA 
acceptance  was  greater  among  personnel  with  lower  skill  levels. 

Regarding  the  type  of  maintenance  tasks  for  which  EPICS  personnel  would  like  to  use 
JPAs,  77  percent  like  to  use  JPAs  the  first  time  they  do  some/most  scheduled  tasks;  81 
percent  liked  using  JPAs  the  first  time  they  do  an  unscheduled  task;  and,  although  80 
percent  prefer  to  use  JPAs  for  at  least  some  infrequently  occurring  maintenance  tasks,  70 
percent  favored  using  a  more  abbreviated  checklist  format  such  as  the  MRC  for 
frequently  occurring  tasks.  Thus,  in  general,  the  EPICS  personnel  preferred  using  JPAs 
for  first  time  and  infrequently  occurring  tasks  but  preferred  MRCs  for  frequently 
occurring  tasks. 

The  decision  to  make  the  JPAs  compatible  with  the  Navy  maintenance  system  may 
have  influenced  the  results.  The  Navy  maintenance  system  requires  documentation  for  all 
scheduled  tasks.  In  contrast,  guidelines  (Joyce,  Chenzoff,  Mulligan,  <5c  Mallory,  1973) 
derived  from  JPA  research  state  that  there  should  be  a  tradeoff  between  the  maintenance 
tasks  supported  with  JPAs  and  the  tasks  supported  with  training.  For  scheduled  tasks  in 
the  the  EPICS  program,  it  was  decided  to  develop  JPAs  that  are  compatible  with  the  Navy 
maintenance  system  even  if  the  task  was  more  suitable  for  training.  Thus,  the  excessive 
detail  of  the  JPAs  may  indeed  provide  all  the  information  necessary  to  perform  tasks 
correctly  the  first  time,  but  continued  exposure  to  such  detail  for  the  frequently 
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occurring  scheduled  tasks  distracts  the  users.  Besides  conforming  strictly  to  the 
JPA/training  tradeoff,  another  method  of  providing  complete  information  in  a  more 
acceptable  format  is  to  use  a  multilevel  JPA.  A  multilevel  JPA  presents  all  the 
information  an  inexperienced  technician  might  need  in  a  format  that  also  allows  an 
experienced  technician  to  use  the  JPA  as  a  checklist  (e.g.,  by  highlighting  critical  steps). 
The  problem  is  then  one  of  cost/benefits;  that  is,  can  the  additional  development  cost  be 
justified? 

EPICS  personnel  may  also  prefer  MRCs  over  JPAs  for  frequent  tasks  because  of  the' 
redundancy  across  similar  tasks.  For  example,  all  the  JPAs  for  a  particular  cabinet 
contain  the  same  access  and  close-up  procedures.  These  portions  could  be  removed  from 
the  JPAs  and  placed  into  a  single  shipboard  instruction  module  to  eliminate  the  constant 
redundancy.  An  integrated  personnel  system  such  as  EPICS  has  this  additional  opportunity 
for  the  JP  A/training  tradeoff. 

These  results  also  agree  with  those  of  Johnson  et  al.  (1977)  for  the  frequently 
occurring  tasks;  that  is,  abbreviated  checklists  are  preferred  over  more  detailed  JPAs. 
For  both  first  time  unscheduled  and  infrequently  occurring  tasks,  however,  Jc  ->on  et  al. 
found  that  the  less  skilled  Air  Force  technicians  liked  JPAs  better  than  di  he  more 
skilled  technicians.  This  difference  is  not  very  evident  in  the  present  study.  In  the  Air 
Force  study,  the  definition  of  skill  level  appears  to  be  a  composite  of  ability,  training,  and 
experience.  All  EPICS  personnel  regardless  of  aptitude  (AFQT  score)  or  FT  eligibility 
(composite  ASVAB  score)  had  the  same  training  and  experience. 

The  open-ended  responses  of  the  shipboard  administrators  and  EPICS  personnel 
support  the  survey  responses  and  provide  some  insight  into  their  specific  likes  and  dislikes. 
The  shipboard  administrators  stated  that  JPAs  provide  good  location  information,  but  the 
format  is  too  simple.  The  EPICS  personnel  also  stated  that  JPAs  (1)  were  useful  for 
performing  the  task  initially,  (2)  provide  helpful  location  information,  and  (3)  organized 
the  technical  information  better  and  provided  better  understanding  of  the  task  than  the 
MRCs.  Negative  comments  from  the  EPICS  personnel  on  the  use  of  JPAs  were  directed 
at  the  frequent  or  familiar  tasks.  Availability  of  only  detailed  JPAs  for  frequently 
occurring  tasks,  as  well  as  problems  in  implementation  may  have  contributed  to  the 
negative  comments;  the  reasons  for  these  will  be  discussed  in  the  Implementation  section. 

Use 


While  measures  of  acceptance  are  used  to  determine  why  a  JPA  is  or  is  not  used, 
measures  of  actual  use  (the  USE  ratios  in  this  study)  provide  information  on  how  and  under 
what  circumstances  a  JPA  is  used.  For  scheduled  tasks,  JPAs  were  used  58  percent  of  the 
time  and,  for  unscheduled  tasks,  JPAs  were  used  81  percent  of  the  time.  Unfortunately, 
there  was  no  way  of  determining  if  the  USE  value  for  the  scheduled  tasks  was  a  result  of 
heavy  JPA  usage  for  first  time  tasks  only,  with  little  or  no  JPA  usage  for  repeat  tasks. 
According  to  the  acceptance  data,  individuals  preferred  not  to  use  JPAs  on  subsequent 
performances  of  frequently  occurring  tasks.  On  the  other  hand,  the  JPA  usage  measures 
for  unscheduled  tasks  support  the  acceptance  findings  of  individuals  preferring  to  use 
JPAs  for  infrequently  occurring  tasks  (most  unscheduled  tasks  have  long  intervals  between 
tasks).  The  high  JPA  usage  also  appears  to  support  the  acceptance  finding  of  using  a  JPA 
the  first  time  a  task  is  performed. 

To  determine  if  the  FT  eligibility  affects  the  use  of  JPAs  (Figure  7),  it  is  important 
to  note  that  FT  ineiigibles  always  used  JPAs  more  than  the  FT  eligibles  (63%  for 
scheduled  tasks  and  86%  for  unscheduled  tasks  compared  to  48%  and  75%  respectively). 
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Only  the  difference  between  scheduled  tasks,  however,  was  significant.  The  FT  eligibles 
also  rated  the  JPAs  as  easy  to  use  and  understand.  These  findings  support  the  assumption 
that  JPAs  were  designed  to  ensure  that  individuals  who  did  not  meet  the  FT-eligibility 
criteria  (i.e.,  with  ASVAB  composite  scores  below  218)  would  be  able  to  perform 
maintenance  using  the  JPAs. 

While  FT  ineligibles  used  3PAs  more,  there  was  no  similar  finding  for  acceptance. 
The  apparent  discrepancy  between  JPA  use  and  acceptance  for  infrequently  occurring 
tasks  is  not  surprising.  The  JPAs  were  designed  to  accommodate  individuals  who  were  not 
eligible  for  the  FT  rating.  They  would  need  the  more  detailed  technical  information  in 
the  JPAs  even  after  performing  a  maintenance  task  for  the  first  time.  On  the  other  hand, 
with  two  levels  of  detail  available  for  the  required  technical  information,  the  FT-eligible 
personnel  would  be  better  able  than  the  FT-ineligible  group  to  use  the  less  detailed  MRCs 
on  subsequent  performances  of  maintenance  tasks.  At  the  same  time,  the  FT-eligible 
group  recognized  their  need  for  the  more  detailed  JPAs  for,  infrequently  occurring  tasks. 

Results  on  JPA  use  for  the  groups  with  high  and  low  AFQT  scores  (a  general  measure 
of  aptitude)  were  similar  to  those  of  the  FT-eligibility  groups.  Individuals  with  the  low 
AFQT  scores  used  JPAs  more  than  individuals  with  the  high  AFQT  scores  (Figure  8). 
However,  unlike  FT  eligibility,  the  comparison  between  the  low  and  high  AFQT  score 
groups  was  significant  for  both  scheduled  and  unscheduled  tasks. 

For  reading  grade  level,  only  the  difference  for  the  unscheduled  tasks  comparison  was 
significant  and  consistent  with  the  findings  for  both  FT  eligibility  and  AFQT  score.  From 
the  past  research  on  JPAs,  it  was  assumed  that  JPAs  with  many  pictorials  would  permit 
individuals  who  did  not  read  well  to  complete  a  given  task  successfully.  It  was  further 
assumed  that,  because  of  the  use  of  many  pictorials,  individuals  with  low  reading  grade 
levels  would  prefer  the  JPA  format  regardless  of  task  type.  This  assumption  was  true 
only  for  unscheduled  tasks. 

Supervisor  ratings  were  correlated  with  the  JPA  usage  measure  for  scheduled  and 
unscheduled  tasks  because  a  relationship  was  predicted  between  the  confidence  a 
supervisor  has  in  an  individual  and  the  percentage  of  time  the  individual  used  JPAs  (Table 
10).  The  negative  correlation  between  the  scheduled  tasks  and  the  supervisor  confidence 
rating  for  the  clean  and  inspect  task  (i.e.,  the  scheduled  task;  r  =  -.39,  p<  .01)  was  the 
only  statistically  significant  coefficient.  This  indicates  that  supervisors  have  little 
confidence  in  an  individual  who  uses  JPAs  a  lot  (i.e.,  the  supervisor  tends  to  view  JPAs  as 
unnecessary  for  scheduled  tasks).  An  alternative  explanation  is  that  supervisors  consider 
the  use  of  JPAs  as  a  necessary  first  step,  but  they  believe  that  to  become  a  competent 
technician,  an  individual  must  go  beyond  the  maintenance  support  documentation  and 
understand  the  purpose  for  doing  preventive  maintenance.  Thus,  a  technician  still  using 
JPAs  may  be  perceived  as  lacking  in  technical  ability.  The  availability  of  a  JPA  with 
multiple  levels  of  detail  m^y  give  rise  to  different  confidence  ratings. 

Another  explanation  for  the  negative  correlation  is  that  supervisors  associated  high 
use  of  JPAs  with  lower  aptitude  personnel  and  that  they  were  aware  of  this  association. 
The  shipboard  administrators'  response  to  a  similar  feedback  survey  itcm--58  percent  had 
little  confidence  in  assigning  EPICS  personnel  who  use  JPAs— supports  either  interpreta¬ 
tion.  The  fact  that  these  correlative  data  are  only  from  a  small  sample  precluded 
examining  differences  according  to  FT  eligibility,  AFQT  score,  or  reading  grade  level. 
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Implementation 

Although  personnel  fleet  assignment  was  included  as  a  variable  in  this  study,  no 
differences  between  JPA  implementation  in  the  Atlantic  and  Pacific  fleets  were 
expected.  However,  both  the  feedback  survey  and  the  JPA  usage  survey  responses 
indicate  differences,  which  the  acceptance  data  and  IMP  values  reflected. 

Comparing  measures  of  acceptance  from  the  feedback  survey,  Atlantic  fleet  person¬ 
nel's  attitudes  were  r^ore  positive  towards  JPAs.  Atlantic  fleet  personnel  (1)  required  less 
help  in  using  JPAs,  (2)  found  JPAs  easier  to  handle,  and  (3)  found  JPA$  easier  to 
understand.  Data  from  the  usage  survey  indicate  a  significant  difference  for  the 
infrequently  occurring  tasks  with  the  Atlantic  fleet  preferring  to  use  JPAs  over  MRCs. 

The  significantly  higher  IMP  value  for  the  Atlantic  fleet  seems  to  indicate  better 
implementation  of  JPAs  in  the  Atlantic  fleet  (i.e.,  the  Atlantic  fleet  personnel  were  more 
willing  to  use  JPAs  for  all  tasks).  Implementation  in  the  Atlantic  fleet  occurred  six 
months  after  implementation  in  the  Pacific  fleet.  Since  the  Pacific  implementation  had 
the  problem  of  JPAs  arriving  after  the  EPICS  personnel  arrived,  the  Atlantic  implemen¬ 
tation  may  have  benefited  from  the  six-month  delay. 

The  overall  IMP  values  indicate  that,  regardless  of  task  category,  less  than  40 
percent  of  the  tasks  performed  included  the  use  of  a  JPA.  Transition  to  MRCs  for 
frequently  occurring  tasks  may  have  accounted  for  some  reduction  in  JPA  use. 


CONCLUSIONS 

1.  The  design  of  the  JPAs  facilitates  acceptance  by  the  users.  EPICS  personnel 
varying  in  FT  eligibility,  general  aptitude,  and  reading  ability  accepted  JPAs  and  used 
them  as  sources  of  information  during  performance  of  maintenance  tasks. 

2.  JPAs  provide  the  most  support  for  initial  performance  of  all  tasks. 

3.  JPAs  meet  the  information  needs  of  EPICS  personnel  who  have  less  aptitude  for 
the  FT  rating  and  have  lower  reading  ability. 


RECOMMENDATIONS 

1.  Regardless  of  existing  maintenance  system  guidelines,  task  frequency  should  be 
incorporated  as  a  design  criterion  for  JPA  development  in  addition  to  the  skill  and 
aptitude  level  of  the  JPA  user.  Any  variable  that  moderates  the  frequency  of  a  task  has 
to  be  considered.  For  example,  a  monthly  scheduled  task  could  be  considered  an 
infrequently  occurring  task  if  work  center  policy  dictates  that  technicians  rotate  task 
duties  in  order  to  gain  experience  with  a  wide  range  of  maintenance  tasks. 

2.  The  following  recommendations  for  future  JPA  implementations  within  an 
integrated  personnel  system  context  are  based  on  experience  gained  from  EPICS: 

a.  JPAs  should  not  compete  with  existing  support  documentation  (e.g.,  MRCs) 
even  if  the  target  audience  has  no  preconceptions  about  which  form  of  maintenance 
documentation  they  prefer.  An  alternative  implementation  strategy  might  be  to  replace 
existing  documentation  with  JPAs.  All  personnel,  both  EPICS  and  existing  work  center 
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personnel,  would  be  shown  how  to  use  JPAs  effectively  for  various  types  of  maintenance 
activities.  On  new  system  developments,  JPA  technology  can  be  employed  at  the  outset 
to  avoid  competing  maintenance  documentation. 

\ 

b.  JPAs  should  be  completely  installed  as  support  documentation  prior  to  the 
arrival  of  the  target  population.  JPA-targeted  personnel  who  report  to  a  site  with 
existing  maintenance  documentation  prior  to  JPA  delivery  will  use  the  existing  docu¬ 
mentation.  This  will  create  another  impediment  to  JPA  acceptance  and  use  when  the 
JPAs  are  installed.  Individuals  are  reluctant  to  change  to  new  documentation  in  a 
different  format  when  they  are  already  familiar  with  the  format  of  the  existing 
documentation. 

c.  Work  center  supervisors  should  support,  the  JPA  implementation  process.  As 
personnel  rotate  frequently  on  Navy  ships,  procedures  and  practices  have  to  be  established 
to  inform  supervisors  of  their  role  in  supporting  the  use  of  JPAs  during  the  implementa¬ 
tion  phase.  This  is  especially  critical  if  maintenance  documentation  in  a  different  format 
already  exists. 

d.  Aptitude,  skill  level,  experience,  task  frequency,  and  task  difficulty  should 
be  integrated  into  JPAs  of  differing  levels  of  detail.  Job  design  should  be  used  to 
establish  a  career  structure  of  two  or  more  levels  of  skill  and  experience  with  a  subset  of 
the  overall  task  domain  represented  at  each  level.  Ideally,  an  electronic  medium  and  a 
digitized  data  base  could  be  used  to  tailor  JPAs  with  multiple  levels  of  detail  individually 
to  the  needs  of  each  user  and  the  type  of  task  (i.e.,  level  of  detail  would  depend  on  the 
skill  level  of  the  user  and/or  the  frequency  of  the  maintenance  task).  Multilevel  JPAs 
make  the  implementation  strategy  more  flexible  because,  regardless  of  career  path,  all 
personnel  could  readily  use  it.  Furthermore,  the  confidence  of  supervisors  in  assigning 
individuals  to  tasks  would  not  be  biased  by  the  type  of  maintenance  documentation  used. 
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